Method and apparatus for generating super night scene image, and electronic device and storage medium

ABSTRACT

The present disclosure discloses a method, device, electronic equipment and storage medium for generating a super night scene image. The method includes the following steps: acquiring consecutive multiple frames of original images, which include a frame of underexposed image and multiple frames of normally exposed images; performing stacked noise reduction processing on the multiple frames of normally exposed images to obtain a frame of normally noise-reduced image; performing gray scale transformation processing on the normally noise-reduced image to obtain a frame of overexposed image; fusing the underexposed image, the normally noise-reduced image and the overexposed image to obtain a frame of super night scene image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Application No.PCT/CN2021/070225, with an international filing date of Jan. 5, 2021,which is based upon and claims priority to Chinese Patent ApplicationNo. 202010014410.7, filed with the Chinese Patent Office on Jan. 7,2020, titled “METHOD AND APPARATUS FOR GENERATING SUPER NIGHT SCENEIMAGE, AND ELECTRONIC DEVICE AND STORAGE MEDIUM”, the entire contents ofwhich are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to the technical field of imageprocessing, and in particular, relates to a method, device, electronicequipment and storage medium for generating a super night scene image.

BACKGROUND

Traditional night scene enhancement photo processing generally uses animage brightness algorithm or a contrast enhancement algorithm forenhancement processing of a night scene image after the image iscaptured, so as to improve the brightness and contrast of the dark partof the image. However, because the night scene image has a lot of noise,and the noise may be enhanced, the enhancement effect of the whole nightscene image is not good. At the same time, if the enhancementcoefficient is determined according to local variance, then halo islikely to occur at the place with excessive intensity of light andshade.

SUMMARY

In one aspect, the present disclosure discloses a method for generatinga super night scene image, and the method includes the following steps:

acquiring consecutive multiple frames of original images, which includea frame of underexposed image and multiple frames of normally exposedimages;

performing stacked noise reduction processing on the multiple frames ofnormally exposed images to obtain a frame of normally noise-reducedimage;

performing gray scale transformation processing on the normallynoise-reduced image to obtain a frame of overexposed image;

fusing the underexposed image, the normally noise-reduced image and theoverexposed image to obtain a frame of super night scene image.

In another aspect, the present disclosure discloses an electronicequipment, which includes a memory, a processor, and a computer programstored in the memory and executable on the processor, and the computerprogram, when executed by the processor, enables the processor toexecute the steps of the method as described above.

In another aspect, the present disclosure discloses a computer-readablestorage medium storing a computer program, and the computer program,when executed by a processor, executes the steps of the method asdescribed above.

BRIEF DESCRIPTION OF THE DRAWINGS

For clearer descriptions of technical solutions according to theembodiments of the present disclosure, drawings that are to be referredfor description of the embodiments are briefly described hereinafter.Apparently, the drawings described hereinafter merely illustrate someembodiments of the present disclosure. Persons of ordinary skill in theart may also derive other drawings based on the drawings describedherein without any creative effort.

FIG. 1 is a flowchart diagram for implementing a method for generating asuper night scene image according to an embodiment of the presentdisclosure.

FIG. 2 is a flowchart diagram for implementing a method for generating asuper night scene image according to a second embodiment of the presentdisclosure.

FIG. 3 is a schematic structural diagram of a device for generating asuper night scene image according to a third embodiment of the presentdisclosure.

FIG. 4 is a schematic structural diagram of an electronic equipmentaccording to a fourth embodiment of the present disclosure.

DETAILED DESCRIPTION

In order to make objectives, technical solutions and advantages of thepresent disclosure clearer, the present disclosure will be furtherdescribed in detail with reference to attached drawings and embodiments.It shall be appreciated that, the specific embodiments described hereinare only used for explaining the present disclosure, and are notintended to limit the present disclosure.

Hereinafter, the specific implementation of the present disclosure willbe described in detail with reference to the specific embodiments.

FIRST EMBODIMENT

FIG. 1 shows a flowchart diagram for implementing a method forgenerating a super night scene image according to an embodiment of thepresent disclosure. For convenience of description, only parts relatedto the embodiment of the present disclosure are shown, and the detailsare as follows.

In step S101, consecutive multiple frames of original images areacquired, and the consecutive multiple frames of original images includea frame of underexposed image and multiple frames of normally exposedimages.

Embodiments of the present disclosure are applicable to electronicequipments, which may include mobile phones, watches, tablets,computers, cameras and other equipments with the photographing function.In embodiments of the present disclosure, multiple frames of originalimages continuously captured by an electronic equipment in a dark place,the dark place is at night or other scenes with dark light, areacquired. For convenience of explanation, in this example, the number offrames of the original images is represented by M, M>3, and the multipleframes of original images should include a frame of underexposed imageand multiple frames of normally exposed images. In practice, for theconvenience of operation, usually the underexposed image is the firstframe of image acquired. For example, if there are 9 frames of originalimages captured continuously, then the first frame of image is theunderexposed image, and the last 8 frames of images are the normallyexposed images.

In step S102, stacked noise reduction processing is performed on themultiple frames of normally exposed images to obtain a frame of normallynoise-reduced image.

In embodiments of the present disclosure, the step of performing stackednoise reduction processing on the multiple frames of normally exposedimages to obtain a frame of normally noise-reduced image includes:

S1021: performing weighted fusion noise reduction processing on themultiple frames of normally exposed images to obtain a frame of stackednoise-reduced image.

In some embodiments, a frame of image is selected from the multipleframes of normally exposed images as a reference image, the images otherthan the reference image in the multiple frames of normally exposedimages are aligned with the reference image, and weighted fusion noisereduction processing is performed on the multiple frames of normallyexposed images that are aligned to obtain the stacked noise-reducedimage, thereby reducing the adverse effect caused by the misalignment ofthe images. Wherein the first frame of image in the normally exposedimages is selected as the reference image, or the second frame of imagein the normally exposed images is selected as the reference image, orthe last frame of image in the normally exposed images is selected asthe reference image, and no limitation is made thereto. The alignment ofa certain frame of image with other frames of images relates to theprior art, e.g., the alignment is performed by sparse optical flow.Therefore, in the alignment of the images other than the reference imagein the multiple frames of normally exposed images with the referenceimage, the alignment is performed by the image alignment methods of theprior art, this will not be further described herein.

Optionally, weighted fusion noise reduction processing is performed onthe multiple frames of normally exposed images that are alignedaccording to a first formula to obtain the stacked noise-reduced image;

wherein the first formula is:

${{\overset{\_}{I}\left( {x,y} \right)} = \frac{\sum\limits_{M - 1}^{I}{{w_{i}\left( {x,y} \right)}{I_{i}\left( {x,y} \right)}}}{\sum\limits_{M - 1}^{I}{w_{i}\left( {x,y} \right)}}},$

Ī(x,y) represents the stacked noise-reduced image, I_(i)(x,y) representsthe i-th image among the multiple frames of normally exposed images thatare aligned, M−1 represents the number of frames of the normally exposedimages, w_(j)(x,y) represents a weight of the weighted fusion, theweight of the weighted fusion is determined by the difference betweenthe current image and the reference image, and the larger the differenceis, the smaller the weight will be. The weight w_(i)(x,y) is calculatedby the following formula:

${w_{i}\left( {x,y} \right)} = \left\{ {\begin{matrix}1 & {{d\left( {x,y} \right)}<=n} \\{\max\left( {\frac{{3n} - {d\left( {x,y} \right)}}{2n},0} \right)} & {{d\left( {x,y} \right)} > n}\end{matrix},} \right.$ d(x, y) = ❘I_(i)(x, y) − I₀(x, y)❘,

wherein I₀(x,y) represents the reference image, d(x,y) represents thebrightness difference value between the i-th image I_(i)(x,y) among theM−1 frames of normally exposed images that are aligned and the referenceimage I₀(x,y), and n represents an intensity value of a preset imagenoise. The noise intensity n is determined by ISO, exposure time of theelectronic equipment, and the properties of the sensor itself. For thesame sensor, ISO represents the photosensitive speed of a CCD or CMOSphotosensitive element, the larger the ISO is, the larger the n will be,and the shorter the exposure time is, the larger the n will be.

In some embodiments, weighted fusion noise reduction processing isperformed on the multiple frames of normally exposed images that arealigned in an RGB space according to the first formula. Optionally,weighted fusion noise reduction processing is performed on the multipleframes of normally exposed images that are aligned in the YUV spaceaccording to the first formula. Optionally, the Y component data of themultiple frames of normally exposed images that are aligned as well asthe U component data and V component data of the reference image areacquired, weighted fusion noise reduction processing is performed on theY component data according to the first formula, and edge-preservingfiltering processing is performed on the U component data and the Vcomponent data. The Y component data after the weighted fusion noisereduction processing and the U component data and the V component dataafter the edge-preserving filtering processing are combined to obtainthe stacked noise-reduced image, thereby improving the calculation speedin the weighted fusion noise reduction process. The edge-preservingfiltering processing is bilateral filtering processing or directionalfiltering processing, and no limitation is made thereto. The stackednoise-reduced image is an image stored in the YUV space, or an imageconverted to the RGB space or other color spaces, and no limitation ismade thereto.

S1022: generating a normally noise-reduced image from the stackednoise-reduced image by a single-frame image noise reduction method.

In some embodiments, generating a normally noise-reduced image from thestacked noise-reduced image by a single-frame image noise reductionmethod, which further decreases the influence of noise. The single-frameimage noise reduction method is performed in a discrete cosine space.When discrete cosine transform is used for noise reduction, one pixel isslid each time. For convenience of description, the corresponding pixelpoint after each sliding is called a target pixel point in someembodiments. Optionally, several reference pixel points are randomlyselected in a neighboring window of a target pixel point, the targetpixel point and each of the reference pixel points are respectivelytaken as centers to obtain pixel blocks; DCT (Discrete Cosine Transform)transform is performed on each of the pixel blocks, and a DCTcoefficient corresponding to each of the pixel blocks is updatedaccording to a preset threshold, DCT inverse transform is performed onthe updated DCT coefficient to reconstruct each of the pixel blocks,weighted averaging is performed on pixel values of pixel pointscorresponding to the position of the target pixel point in each of thereconstructed pixel blocks, and the pixel value obtained after theweighted averaging is taken as the pixel value of the target pixelpoint, thereby effectively improving the noise reduction effect of theimage. Wherein the size of the neighboring window is usually 8×8, andcorrespondingly, the size of each of the pixel blocks is usually 8×8.The preset threshold is set manually, and the value of the threshold isset according to the noise level of the image. The noise level isdetermined by ISO, exposure time of the camera and sensor of the cameraitself. The higher the value of ISO is, the stronger the photosensitivecapacity of the photosensitive element will be. When the DCT coefficientcorresponding to each pixel block is updated according to the presetthreshold, optionally, the coefficient smaller than the preset thresholdamong the DCT coefficients is set to be zero.

In step S103, gray scale transformation processing is performed on thenormally noise-reduced image to obtain a frame of overexposed image.

In some embodiments of the present disclosure, in the process ofperforming gray scale transformation processing on the normallynoise-reduced image, inverse transformation processing, logarithmictransformation processing or piecewise linear transformation processingis performed on the normally noise-reduced image. Optionally, gammatransformation processing is performed on the normally noise-reducedimage to obtain an overexposed image with enhanced contrast anddistinguishable details. In some embodiments, the formula of gammatransformation is s=cr^(γ), wherein s represents the overexposed image,r represents the gray scale value of the normally noise-reduced imagewhich ranges from [0, 1], c represents the gray scale scalingcoefficient which is used to stretch the image gray scale on the whole.Optionally, the value of c is 1. γ represents the gamma coefficient, andoptionally, γ=0.625 so as to improve the enhancement effect for theimage.

In step S104, the underexposed image, the normally noise-reduced imageand the overexposed image are fused to obtain a frame of super nightscene image.

In some embodiments of the present disclosure, the underexposed imagecan provide rich details for the high light in the image, the normallynoise-reduced image can provide rich details for the middle brightnessin the image, and the overexposed image can provide details for the darkpart of the image. By fusing the underexposed image, the normallynoise-reduced image and the overexposed image, a fused super night sceneimage is obtained, and the super night scene image is an HDR (HighDynamic Range) image. In the operation of fusing the underexposed image,the normally noise-reduced image and the overexposed image, optionally,motion pixel removal is performed on the underexposed image and theoverexposed image with reference to the normally exposed image so as toobtain a third image and a fourth image, respectively; down-sampling isperformed on the normally exposed image, the third image and the fourthimage to calculate a first weight map, three images obtained by thedown-sampling are respectively converted into gray scale images, andmulti-resolution fusion is performed on the gray scale images tocalculate a second weight map. The second weight image is up-sampled tothe same size as the original image, and weighted fusion is performed onthe normally exposed image, the underexposed image and the overexposedimage, thereby improving the visual effect of the HDR image. In someembodiments, other fusion processing methods in the prior art may alsobe used to fuse the underexposed image, the normally noise-reduced imageand the overexposed image, and this will not be further describedherein.

In some embodiments of the present disclosure, acquiring consecutivemultiple frames of original images, which include a frame ofunderexposed image and multiple frames of normally exposed images; inresponse to performing stacked noise reduction processing on themultiple frames of normally exposed images to obtain a frame of normallynoise-reduced image; in response to performing gray scale transformationprocessing on the normally noise-reduced image to obtain a frame ofoverexposed image; and fusing the underexposed image, the normallynoise-reduced image and the overexposed image to obtain a frame of supernight scene image. As such, the noise of the night scene images isdecreased and the user experience is improved. In the stacked noisereduction processing for the multiple frames of normally exposed images,weighted fusion noise reduction processing is performed on the multipleframes of normally exposed images that are aligned in the YUV space,thereby improving the processing speed for the night scene image, andfurther improving the user experience.

SECOND EMBODIMENT

FIG. 2 shows a flowchart diagram for implementing a method forgenerating a super night scene image according to a second embodiment ofthe present disclosure. For convenience of description, only partsrelated to the embodiment of the present disclosure are shown, and thedetails are as follows.

In step S201, consecutive multiple frames of original images, whichinclude a frame of underexposed image and multiple frames of normallyexposed images, are acquired.

In step S202, stacked noise reduction processing is performed on themultiple frames of normally exposed images to obtain a frame of normallynoise-reduced image.

In step S203, gray scale transformation processing is performed on thenormally noise-reduced image to obtain a frame of overexposed image.

In step S204, the underexposed image, the normally noise-reduced imageand the overexposed image are fused to obtain a frame of super nightscene image.

In the embodiment of the present disclosure, reference is made to thedescription of steps S101 to S104 in the first embodiment describedabove for the implementation of steps S201 to S204, and this will not befurther described herein.

In step S205, detail enhancement processing is performed on the supernight scene image by a detail enhancement algorithm to obtain a supernight scene image after the detail enhancement.

In the embodiment of the present disclosure, detail enhancementprocessing is performed on the super night scene image by the detailenhancement algorithm to obtain a super night scene image after thedetail enhancement, thereby further improving the details of the imageand further improving the definition of the image.

Optionally, the detail enhancement algorithm is a detail enhancementalgorithm based on edge-preserving filtering, and the super night sceneimage after the detail enhancement is: I′(x,y)=k₁I(x,y)+(1−k₁)S₁(x,y),wherein I′(x,y) represents the super night scene image after the detailenhancement, I(x,y) represents the super night scene image, S₁(x,y)represents an image obtained by edge-preserving filtering processing onthe super night scene image, represents the coefficient for image detailenhancement, and k₁>1. Specifically, edge-preserving filteringprocessing is performed on the super night scene image I(x,y) to obtainthe image S₁(x,y) after edge-preserving filtering processing, and thesuper night scene image after the detail enhancement can be obtainedthrough the formula

I′(x,y)=k ₁ I(x,y)+(1−k ₁)S ₁(x,y).

Optionally, the detail enhancement algorithm is a detail enhancementalgorithm based on ordinary filtering, and the super night scene imageafter the detail enhancement is:I′(x,y)=max(min(T(x,y),I_(d)(x,y)),I_(e)(x,y)), whereinT(x,y)=k₂I(x,y)+(1−k₂)S₂(x,y), I′(x,y) represents the super night sceneimage after the detail enhancement, I(x,y) represents the super nightscene image, S₂(x,y) represents an image obtained after performingordinary filtering processing on the super night scene image, representsan image obtained after performing expansion processing on the supernight scene image, I_(e)(x,y) represents an image obtained afterperforming corrosion processing on the super night scene image, k₂represents the coefficient for image detail enhancement, and k₂>1.Specifically, ordinary filtering processing, expansion processing andcorrosion processing are respectively performed on the super night sceneimage I(x,y) to obtain the image S₂(x,y) after the ordinary filteringprocessing, the image after the expansion processing, and the imageI_(e)(x,y) after the corrosion processing, and then the super nightscene image after the detail enhancement can be obtained according tothe formula:

I′(x,y)=max(min(T(x,y),I _(d)(x,y)),I _(e)(x,y))

T(x,y)=k ₂ I(x,y)+(1−k ₂)S ₂(x,y).

In some embodiments of the present disclosure, acquiring consecutivemultiple frames of original images, which include a frame ofunderexposed image and multiple frames of normally exposed images; inresponse to performing stacked noise reduction processing on themultiple frames of normally exposed images to obtain a frame of normallynoise-reduced image; in response to performing gray scale transformationprocessing on the normally noise-reduced image to obtain a frame ofoverexposed image; in response to fusing the underexposed image, thenormally noise-reduced image and the overexposed image to obtain a frameof super night scene image; and preforming detail enhancement processingon the super night scene image by a detail enhancement algorithm. Assuch, the details of the night scene images and the user experience areboth improved.

THIRD EMBODIMENT

FIG. 3 shows the structure of a device for generating a super nightscene image according to the third embodiment of the present disclosure.For convenience of description, only parts related to the embodiment ofthe present disclosure are shown, and the device includes:

an image acquisition unit 31, being configured to acquire consecutivemultiple frames of original images, which include a frame ofunderexposed image and multiple frames of normally exposed images;

a stacked noise reduction unit 32, being configured to perform stackednoise reduction processing on the multiple frames of normally exposedimages to obtain a frame of normally noise-reduced image;

a gray scale transformation unit 33, being configured to perform grayscale transformation processing on the normally noise-reduced image toobtain a frame of overexposed image; and

an image synthesis unit 34, being configured to fuse the underexposedimage, the normally noise-reduced image and the overexposed image toobtain a frame of super night scene image.

Optionally, the stacked noise reduction unit includes:

a first noise reduction subunit, being configured to perform weightedfusion noise reduction processing on the multiple frames of normallyexposed images to obtain a frame of stacked noise-reduced image;

a second noise reduction subunit, being configured to perform noisereduction processing on the stacked noise-reduced image by asingle-frame image noise reduction method to obtain the normallynoise-reduced image.

Optionally, the first noise reduction subunit includes:

a reference image selection unit, being configured to select a frame ofimage from the multiple frames of normally exposed images as a referenceimage;

an image alignment processing unit, being configured to align the imagesother than the reference image among the multiple frames of normallyexposed images with the reference image;

a third noise reduction subunit, being configured to perform weightedfusion noise reduction processing on the multiple frames of normallyexposed images that are aligned according to the first formula to obtainthe stacked noise-reduced image. The first formula is:

${{\overset{\_}{I}\left( {x,y} \right)} = \frac{\sum\limits_{M - 1}^{I}{{w_{i}\left( {x,y} \right)}{I_{i}\left( {x,y} \right)}}}{\sum\limits_{M - 1}^{I}{w_{i}\left( {x,y} \right)}}},$

wherein Ī(x,y) represents the stacked noise-reduced image, I_(i)(x,y)represents the i-th image among the multiple frames of normally exposedimages that are aligned, M−1 represents the number of frames of thenormally exposed images, w_(i)(x,y) represents a weight of the weightedfusion, and the weight w_(i)(x,y) is calculated by the followingformula:

${w_{i}\left( {x,y} \right)} = \left\{ {\begin{matrix}1 & {{d\left( {x,y} \right)}<=n} \\{\max\left( {\frac{{3n} - {d\left( {x,y} \right)}}{2n},0} \right)} & {{d\left( {x,y} \right)} > n}\end{matrix},} \right.$ d(x, y) = ❘I_(i)(x, y) − I₀(x, y)❘,

wherein I₀(x,y) represents the reference image, and n represents anintensity value of a preset image noise.

Optionally, the second noise reduction subunit includes:

a pixel point selecting unit, being configured to randomly selectseveral reference pixel points in a neighboring window of a target pixelpoint;

a pixel block acquisition unit, being configured to respectively takethe target pixel point and each of the reference pixel points as centersto obtain pixel blocks;

a DCT transform unit, being configured to perform DCT transform on eachof the pixel blocks, and update a DCT coefficient corresponding to eachof the pixel blocks according to a preset threshold;

an DCT inverse transform unit, being configured to perform DCT inversetransform on the updated DCT coefficient to reconstruct each of thepixel blocks;

a pixel value calculating unit, being configured to perform weightedaveraging on pixel values of pixel points corresponding to the positionof the target pixel point in each of the reconstructed pixel blocks, andtake the pixel value obtained after the weighted averaging as the pixelvalue of the target pixel point.

Optionally, the DCT transform unit includes:

a DCT transform subunit, being configured to set the coefficient smallerthan the preset threshold among the DCT coefficients to be zero.

Optionally, the third noise reduction subunit includes:

a fourth noise reduction subunit, being configured to perform weightedfusion noise reduction processing on the multiple frames of normallyexposed images that are aligned in a YUV space according to the firstformula.

Optionally, the fourth noise reduction subunit includes:

a component data acquisition unit, being configured to acquire Ycomponent data of the multiple frames of normally exposed images thatare aligned, and U component data and V component data of the referenceimage; or

a component data acquisition unit, being configured to perform weightedfusion noise reduction processing on the Y component data and performingedge-preserving filtering processing on the U component data and the Vcomponent data according to the first formula;

a component data synthesis unit, being configured to combine the Ycomponent data after the weighted fusion noise reduction processing, andthe U component data and the V component data after the edge-preservingfiltering processing to obtain the stacked noise-reduced image.

Optionally, the gray scale transformation unit includes:

a gamma transformation unit, being configured to perform gammatransformation processing on the normally noise-reduced image.

Optionally, the gamma coefficient used for gamma transformationprocessing on the normally noise-reduced image is 0.625.

Optionally, the device further includes:

a detail enhancement processing unit, being configured to perform detailenhancement processing on the super night scene image by a detailenhancement algorithm to obtain a super night scene image after thedetail enhancement;

wherein the detail enhancement algorithm is a detail enhancementalgorithm based on edge-preserving filtering, and the super night sceneimage after the detail enhancement is: I′(x,y)=k₁I(x,y)+(1−k₁)S₁(x,y),wherein I′(x,y) represents the super night scene image after the detailenhancement, I(x,y) represents the super night scene image, S₁(x,y)represents an image obtained by edge-preserving filtering processing onthe super night scene image, k₁ represents the coefficient for imagedetail enhancement, and k₁>1; or

-   -   the detail enhancement algorithm is a detail enhancement        algorithm based on ordinary filtering, and the super night scene        image after the detail enhancement is:

I′(x,y)=max(min(T(x,y),I _(d)(x,y)),I _(e)(x,y)), wherein

T(x,y)=k₂I(x,y)+(1−k₂)S(x,y), I′(x,y) represents the super night sceneimage after the detail enhancement, I(x,y) represents the super nightscene image, S₂(x,y) represents an image obtained after performingordinary filtering processing on the super night scene image,I_(d)(x,y), represents an image obtained after performing expansionprocessing on the super night scene image, I_(e)(x,y) represents animage obtained after performing corrosion processing on the super nightscene image, represents the coefficient for image detail enhancement,and k₂>1.

In some embodiments of the present disclosure, units of the device forgenerating the super night scene image is implemented by correspondinghardware or software units, and each of the units is an independentsoftware or hardware unit or integrated into one software or hardwareunit, and this is not intended to limit the present disclosure.Reference is made to the description of the embodiments of the methoddescribed above for the specific implementation of each of the units ofthe device for generating the super night scene image, and this will notbe further described herein.

FOURTH EMBODIMENT

FIG. 4 shows the structure of an electronic equipment according to thefourth embodiment of the present disclosure. For convenience ofdescription, only parts related to the embodiment of the presentdisclosure are shown.

An electronic equipment 4 of the embodiment of the present disclosureincludes a processor 40, a memory 41, and a computer program 42 storedin the memory 41 and executable on the processor 40. The computerprogram 42, when executed by the processor 40, enables the processor 40to execute the steps of the method for generating a super night sceneimage according to the embodiments described above, e.g., steps S101 toS104 shown in FIG. 1 . Alternatively, the computer program 42, whenexecuted by the processor 40, enables the processor 40 to implement thefunctions of the units in the embodiments of the device described above,e.g., functions of the units 31 to 34 shown in FIG. 3 .

In some embodiments of the present disclosure, acquiring consecutivemultiple frames of original images, which include a frame ofunderexposed image and multiple frames of normally exposed images; inresponse to performing stacked noise reduction processing on themultiple frames of normally exposed images to obtain a frame of normallynoise-reduced image; in response to performing gray scale transformationprocessing on the normally noise-reduced image to obtain a frame ofoverexposed image; and fusing the underexposed image, the normallynoise-reduced image and the overexposed image to obtain a frame of supernight scene image. As such, the noise of the night scene images isdecreased and the user experience is improved.

FIFTH EMBODIMENT

In some embodiments of the present disclosure, a computer-readablestorage medium is provided, and the computer-readable storage mediumstores a computer program which, when executed by a processor, executesthe steps of the method for generating a super night scene imageaccording to the embodiments described above, e.g., steps S101 to S104shown in FIG. 1 . Alternatively, the computer program, when executed bythe processor, implements the functions of the units of the deviceaccording to the embodiments described above, e.g., functions of theunits 31 to 34 shown in FIG. 3 .

In some embodiments of the present disclosure, acquiring consecutivemultiple frames of original images, which include a frame ofunderexposed image and multiple frames of normally exposed images; inresponse to performing stacked noise reduction processing on themultiple frames of normally exposed images to obtain a frame of normallynoise-reduced image; in response to performing gray scale transformationprocessing on the normally noise-reduced image to obtain a frame ofoverexposed image; and fusing the underexposed image, the normallynoise-reduced image and the overexposed image to obtain a frame of supernight scene image. As such, the noise of the night scene images isdecreased and the user experience is improved.

The computer-readable storage medium of the embodiment of the presentdisclosure include any entity or device, recording medium capable ofcarrying computer program code, such as an ROM/RAM, a magnetic disk, anoptical disk, a flash memory and other memories.

What described above are only preferred embodiments of the presentdisclosure, and are not intended to limit the present disclosure. Anymodifications, equivalent substitutions and improvements made within thespirit and principle of the present disclosure shall be included in thescope claimed in the present disclosure.

1. A method for generating a super night scene image, beingcharacterized in that, the method comprising the following steps:acquiring consecutive multiple frames of original images, which includea frame of underexposed image and multiple frames of normally exposedimages; performing stacked noise reduction processing on the multipleframes of normally exposed images to obtain a frame of normallynoise-reduced image; performing gray scale transformation processing onthe normally noise-reduced image to obtain a frame of overexposed image;fusing the underexposed image, the normally noise-reduced image and theoverexposed image to obtain a frame of super night scene image.
 2. Themethod according to claim 1, wherein the step of performing stackednoise reduction processing on the multiple frames of normally exposedimages to obtain a frame of normally noise-reduced image comprises:performing weighted fusion noise reduction processing on the multipleframes of normally exposed images to obtain a frame of stackednoise-reduced image; performing noise reduction processing on thestacked noise-reduced image by a single-frame image noise reductionmethod to obtain the normally noise-reduced image.
 3. The methodaccording to claim 2, wherein the step of performing weighted fusionnoise reduction processing on the multiple frames of normally exposedimages to obtain a frame of stacked noise-reduced image comprises:selecting a frame of image from the multiple frames of normally exposedimages as a reference image; aligning the images other than thereference image in the multiple frames of normally exposed images withthe reference image; performing weighted fusion noise reductionprocessing on the multiple frames of normally exposed images that arealigned according to a first formula to obtain the stacked noise-reducedimage; wherein the first formula is${{\overset{\_}{I}\left( {x,y} \right)} = \frac{\sum\limits_{M - 1}^{I}{{w_{i}\left( {x,y} \right)}{I_{i}\left( {x,y} \right)}}}{\sum\limits_{M - 1}^{I}{w_{i}\left( {x,y} \right)}}},$Ī(x,y) represents the stacked noise-reduced image, I_(i)(x,y) representsthe i-th image among the multiple frames of normally exposed images thatare aligned, M−1 represents the number of frames of the normally exposedimages, w_(i)(x,y) represents a weight of the weighted fusion, and theweight w_(i)(x,y) is calculated by the following formula:$\underline{{w_{i}\left( {x,y} \right)} = \left\{ {\begin{matrix}1 & {{d\left( {x,y} \right)}<=n} \\{\max\left( {\frac{{3n} - {d\left( {x,y} \right)}}{2n},0} \right)} & {{d\left( {x,y} \right)} > n}\end{matrix},} \right.}$$\underline{{{d\left( {x,y} \right)} = {❘{{I_{i}\left( {x,y} \right)} - {I_{0}\left( {x,y} \right)}}❘}},}$wherein I₀(x,y) represents the reference image, and n represents anintensity value of a preset image noise.
 4. The method according toclaim 3, wherein the step of performing weighted fusion noise reductionprocessing on the multiple frames of normally exposed images that arealigned according to a first formula comprises: performing weightedfusion noise reduction processing on the multiple frames of normallyexposed images that are aligned in a YUV space according to the firstformula.
 5. The method according to claim 4, wherein the step ofperforming weighted fusion noise reduction processing on the multipleframes of normally exposed images that are aligned in a YUV spaceaccording to the first formula comprises: acquiring Y component data ofthe multiple frames of normally exposed images that are aligned, and Ucomponent data and V component data of the reference image; performingweighted fusion noise reduction processing on the Y component data andperforming edge-preserving filtering processing on the U component dataand the V component data according to the first formula; combining the Ycomponent data after the weighted fusion noise reduction processing, andthe U component data and the V component data after the edge-preservingfiltering processing to obtain the stacked noise-reduced image.
 6. Themethod according to claim 2, wherein the step of performing noisereduction processing on the stacked noise-reduced image by asingle-frame image noise reduction method comprises: randomly selectingseveral reference pixel points in a neighboring window of a target pixelpoint; respectively taking the target pixel point and each of thereference pixel points as centers to obtain pixel blocks; performing DCTtransform on each of the pixel blocks, and updating a DCT coefficientcorresponding to each of the pixel blocks according to a presetthreshold; performing DCT inverse transform on the updated DCTcoefficient to reconstruct each of the pixel blocks; performing weightedaveraging on pixel values of pixel points corresponding to the positionof the target pixel point in each of the reconstructed pixel blocks, andtaking the pixel value obtained after the weighted averaging as thepixel value of the target pixel point.
 7. The method according to claim6, wherein the step of updating a DCT coefficient corresponding to eachof the pixel blocks according to a preset threshold comprises: settingthe coefficient smaller than the preset threshold among the DCTcoefficients to be zero.
 8. The method according to claim 1, wherein thestep of performing gray scale transformation processing on the normallynoise-reduced image comprises: performing gamma transformationprocessing on the normally noise-reduced image.
 9. The method accordingto claim 8, wherein the gamma coefficient used for gamma transformationprocessing on the normally noise-reduced image is 0.625.
 10. The methodaccording to claim 1, wherein after the step of fusing the underexposedimage, the normally noise-reduced image and the overexposed image toobtain a frame of super night scene image, the method further comprises:performing detail enhancement processing on the super night scene imageby a detail enhancement algorithm to obtain a super night scene imageafter the detail enhancement; wherein the detail enhancement algorithmis a detail enhancement algorithm based on edge-preserving filtering,and the super night scene image after the detail enhancement is:I′(x,y)=k₁I(x,y)+(1−k₁)S₁(x,y), wherein I′(x,y) represents the supernight scene image after the detail enhancement, I(x,y) represents thesuper night scene image, S₁(x,y) represents an image obtained afterperforming edge-preserving filtering processing on the super night sceneimage, k₁ represents the coefficient for image detail enhancement, andk₁>1; or the detail enhancement algorithm is a detail enhancementalgorithm based on ordinary filtering, and the super night scene imageafter the detail enhancement is:I′(x,y)=max(min(T(x,y),I _(d)(x,y)),I _(e)(x,y)), whereinT(x,y)=k₂I(x,y)+(1−k₂)S₂(x,y), I′(x,y) represents the super night sceneimage after the detail enhancement, I(x,y) represents the super nightscene image, S₂(x,y) represents an image obtained after performingordinary filtering processing on the super night scene image, I_(d)(x,y)represents an image obtained after performing expansion processing onthe super night scene image, I_(e)(x,y) represents an image obtainedafter performing corrosion processing on the super night scene image, k₂represents the coefficient for image detail enhancement, and k₂>1. 11.(canceled)
 12. An electronic equipment, comprising: a processor; memoryin electronic communication with the processor; one or more computerprograms, stored in the memory and configured to be executed by theprocessor, wherein the computer program, when executed by the processor,enabling the processor to execute the method for generating a supernight scene image; wherein the method for generating a super night sceneimage comprises: acquiring consecutive multiple frames of originalimages, which include a frame of underexposed image and multiple framesof normally exposed images; performing stacked noise reductionprocessing on the multiple frames of normally exposed images to obtain aframe of normally noise-reduced image; performing gray scaletransformation processing on the normally noise-reduced image to obtaina frame of overexposed image; fusing the underexposed image, thenormally noise-reduced image and the overexposed image to obtain a frameof super night scene image.
 13. A computer-readable storage mediumstoring a computer program, wherein the computer program, when beingexecuted by a processor, executes a method for generating a super nightscene image; wherein the method for generating a super night scene imagecomprises: acquiring consecutive multiple frames of original images,which include a frame of underexposed image and multiple frames ofnormally exposed images; performing stacked noise reduction processingon the multiple frames of normally exposed images to obtain a frame ofnormally noise-reduced image; performing gray scale transformationprocessing on the normally noise-reduced image to obtain a frame ofoverexposed image; fusing the underexposed image, the normallynoise-reduced image and the overexposed image to obtain a frame of supernight scene image.
 14. The electronic equipment according to claim 11,wherein the step of performing stacked noise reduction processing on themultiple frames of normally exposed images to obtain a frame of normallynoise-reduced image comprises: performing weighted fusion noisereduction processing on the multiple frames of normally exposed imagesto obtain a frame of stacked noise-reduced image: performing noisereduction processing on the stacked noise-reduced image by asingle-frame image noise reduction method to obtain the normallynoise-reduced image.
 15. The electronic equipment according to claim 12,wherein the step of performing weighted fusion noise reductionprocessing on the multiple frames of normally exposed images to obtain aframe of stacked noise-reduced image comprises: selecting a frame ofimage from the multiple frames of normally exposed images as a referenceimage; aligning the images other than the reference image in themultiple frames of normally exposed images with the reference image;performing weighted fusion noise reduction processing on the multipleframes of normally exposed images that are aligned according to a firstformula to obtain the stacked noise-reduced image; wherein the firstformula is${{\overset{\_}{I}\left( {x,y} \right)} = \frac{\sum\limits_{M - 1}^{I}{{w_{i}\left( {x,y} \right)}{I_{i}\left( {x,y} \right)}}}{\sum\limits_{M - 1}^{I}{w_{i}\left( {x,y} \right)}}},$ Ī(x,y) represents the stacked noise-reduced image, I_(i)(x,y)represents the i-th image among the multiple frames of normally exposedimages that are aligned, M−1 represents the number of frames of thenormally exposed images, w_(i)(x,y) represents a weight of the weightedfusion, and the weight w_(i)(x,y) is calculated by the followingformula:$\underline{{w_{i}\left( {x,y} \right)} = \left\{ {\begin{matrix}1 & {{d\left( {x,y} \right)}<=n} \\{\max\left( {\frac{{3n} - {d\left( {x,y} \right)}}{2n},0} \right)} & {{d\left( {x,y} \right)} > n}\end{matrix},} \right.}$$\underline{{{d\left( {x,y} \right)} = {❘{{I_{i}\left( {x,y} \right)} - {I_{0}\left( {x,y} \right)}}❘}},}$ wherein I₀(x,y) represents the reference image, and n represents anintensity value of a preset image noise.
 16. The electronic equipmentaccording to claim 13, wherein the step of performing weighted fusionnoise reduction processing on the multiple frames of normally exposedimages that are aligned according to a first formula comprises:performing weighted fusion noise reduction processing on the multipleframes of normally exposed images that are aligned in a YUV spaceaccording to the first formula.
 17. The electronic equipment accordingto claim 14, wherein the step of performing weighted fusion noisereduction processing on the multiple frames of normally exposed imagesthat are aligned in a YUV space according to the first formulacomprises: acquiring Y component data of the multiple frames of normallyexposed images that are aligned, and U component data and V componentdata of the reference image; performing weighted fusion noise reductionprocessing on the Y component data and performing edge-preservingfiltering processing on the U component data and the V component dataaccording to the first formula; combining the Y component data after theweighted fusion noise reduction processing, and the U component data andthe V component data after the edge-preserving filtering processing toobtain the stacked noise-reduced image.
 18. The electronic equipmentaccording to claim 12, wherein the step of performing noise reductionprocessing on the stacked noise-reduced image by a single-frame imagenoise reduction method comprises: randomly selecting several referencepixel points in a neighboring window of a target pixel point;respectively taking the target pixel point and each of the referencepixel points as centers to obtain pixel blocks; performing DCT transformon each of the pixel blocks, and updating a DCT coefficientcorresponding to each of the pixel blocks according to a presetthreshold; performing DCT inverse transform on the updated DCTcoefficient to reconstruct each of the pixel blocks; performing weightedaveraging on pixel values of pixel points corresponding to the positionof the target pixel point in each of the reconstructed pixel blocks, andtaking the pixel value obtained after the weighted averaging as thepixel value of the target pixel point.
 19. The electronic equipmentaccording to claim 16, wherein the step of updating a DCT coefficientcorresponding to each of the pixel blocks according to a presetthreshold comprises: setting the coefficient smaller than the presetthreshold among the DCT coefficients to be zero.
 20. The electronicequipment according to claim 11, wherein the step of performing grayscale transformation processing on the normally noise-reduced imagecomprises: performing gamma transformation processing on the normallynoise-reduced image.
 21. The electronic equipment according to claim 11,wherein after the step of fusing the underexposed image, the normallynoise-reduced image and the overexposed image to obtain a frame of supernight scene image, the method further comprises: performing detailenhancement processing on the super night scene image by a detailenhancement algorithm to obtain a super night scene image after thedetail enhancement; wherein the detail enhancement algorithm is a detailenhancement algorithm based on edge-preserving filtering, and the supernight scene image after the detail enhancement is:I′(x,y)=k₁(x,y)+(1−k₁)S₁(x,y), wherein I′(x,y) represents the supernight scene image after the detail enhancement, I(x,y) represents thesuper night scene image, S₁(x,y) represents an image obtained afterperforming edge-preserving filtering processing on the super night sceneimage, k₁ represents the coefficient for image detail enhancement, andk₁>1; or the detail enhancement algorithm is a detail enhancementalgorithm based on ordinary filtering, and the super night scene imageafter the detail enhancement is:I′(x,y)=max(min(T(x,y),I_(d)(x,y)),I_(e)(x,y)), whereinT(x,y)=k₂I(x,y)+(1−k₂)S₂(x,y), I′(x,y) represents the super night sceneimage after the detail enhancement, I(x,y) represents the super nightscene image, S₂(x,y) represents an image obtained after performingordinary filtering processing on the super night scene image, I_(d)(x,y)represents an image obtained after performing expansion processing onthe super night scene image, I_(e)(x,y) represents an image obtainedafter performing corrosion processing on the super night scene image, k₂represents the coefficient for image detail enhancement, and k₂>1.